In one computerized image processing application, the three-dimensional (3-D) shape of a structure can be recovered from a sequence of digitized images taken by a camera. Typically, one of the first steps taken prior to the actual 3-D reconstruction of an object is a process of camera calibration.
Therefore, particularly germane to 3-D reconstruction using an uncalibrated camera is self-calibration. Self-calibration refers to a recovery of camera parameters based only on correspondences of images taken at different object (or camera) poses. Most of the work done on self-calibration relies on knowing the motion of the camera (or object), such as pure translational motion, known camera rotation angles, and pure rotation about the center of the camera.
The traditional approach to 3-D reconstruction with multiple images using an uncalibrated camera applies affine and projective reconstruction techniques. The traditional approach to reconstruct scaled Euclidean structure is usually from known camera parameters.
For example, some techniques recover the Euclidean structure from structure rotation or equivalently, camera orbital motion under the assumption that the camera parameters and structure motion are known. Other techniques use a grided annulus pattern inside which the structure is placed. Camera parameters are extracted from the detected pattern, and the 3-D shape of the structure is recovered from silhouettes. Other known techniques use a closed-form solution for scaled Euclidean reconstruction with known intrinsic camera parameters but unknown extrinsic camera parameters. However, these techniques assume the existence of four coplanar correspondences that are not necessarily collinear.
Recently, 3-D scaled Euclidean structures have been reconstructed from images using an uncalibrated camera. For example, one method reconstructs a scaled Euclidean object under constant but unknown intrinsic camera parameters. There, a minimum of three images are required to recover a unique solution to the intrinsic camera parameters and the Euclidean structure.
The method is based on recovering an intermediate projective structure. Then, an optimization formulation that is based on the Frobenius norm of a matrix is applied. However, this is not equivalent to the more optimal metric of minimizing feature location errors in a 2-D image space. This method has been extended to show that scaled Euclidean reconstruction under a known image aspect ratio and skew, but varying and unknown focal length and principal point is also possible.
The assumption there is that the camera is undergoing general motion. It has been shown that it is not possible to reconstruct a scaled Euclidean structure under constrained motion such as pure translational or orbital motion.
In a two-step approach, a scaled Euclidean structure is recovered from a multiple image sequence with unknown but constant intrinsic parameters. The first stage involves affine camera parameter recovery using the so-called modulus constraint. This is followed in a second stage by conversion to scaled Euclidean structure. This approach can be extended to remove the assumption of a fixed camera focal length.
One known technique provides a method for camera self-calibration from several views. There a two-step approach is also used to recover a scaled Euclidean structure. The method first recovers projective structure before applying a heuristic search to extract the five intrinsic camera parameters. The heuristic involves iterating over several sets of initialization values and periodically checking for convergence.
A detailed characterization of critical motion sequences (CMS) has been given to account for ambiguities in reconstructing a 3-D structure. A critical motion sequence is a camera motion sequence that results in ambiguities in 3-D reconstruction when camera parameters are unknown. For example, only affine structures can be extracted from pure camera translational motion. Of particular relevance to the present disclosure is a determination that there is two degree of freedom projective ambiguity for orbital motions, i.e., pure rotation about a fixed arbitrary axis. In other words, it has been demonstrated that there exists a 2 degree-of-freedom ambiguity in scaled Euclidean reconstruction.
There are three possible options in recovering scaled Euclidean structure from orbital motion:
(1) fix two intrinsic camera parameters; PA1 (2) impose structural constraints, e.g., orthogonality, parallelism, known 3-D locations of fiducial points; or PA1 (3) get the "best" reconstruction without (1) or (2) as disclosed herein.